On the Statistical Estimation of Rényi Entropies
نویسندگان
چکیده
Estimating entropies is important in many fields including statistical physics, machine learning and statistics. While the Shannon logarithmic entropy is the most fundamental, other Rényi entropies are also of importance. In this paper, we derive a bias corrected estimator for a subset of Rényi entropies. The advantage of the estimator is demonstrated via theoretical and experimental considerations.
منابع مشابه
Kernelized Rényi distance for subset selection and similarity scoring
Rényi entropy refers to a generalized class of entropies that have been used in several applications. In this work, we derive a non-parametric distance between distributions based on the quadratic Rényi entropy. The distributions are estimated via Parzen density estimates. The quadratic complexity of the distance evaluation is mitigated with GPUbased parallelization. This results in an efficien...
متن کاملA duality relation connecting different quantum generalizations of the conditional Rényi entropy
Recently a new quantum generalization of the Rényi divergence and the corresponding conditional Rényi entropies was proposed. Here we report on a surprising relation between conditional Rényi entropies based on this new generalization and conditional Rényi entropies based on the quantum relative Rényi entropy that was used in previous literature. This generalizes the well-known duality relation...
متن کاملα-z-Rényi relative entropies
We consider a two-parameter family of Rényi relative entropies Dα,z(ρ||σ) that are quantum generalisations of the classical Rényi divergence Dα(p||q). This family includes many known relative entropies (or divergences) such as the quantum relative entropy, the recently defined quantum Rényi divergences, as well as the quantum Rényi relative entropies. All its members satisfy the quantum general...
متن کاملGeneralized entropies through Bayesian estimation
The demand made upon computational analysis of observed symbolic sequences has been increasing in the last decade. Here, the concept of entropy receives applications, and the generalizations according to Tsallis H (T) q and R enyi H (R) q provide whole-spectra of entropies characterized by an order q. An enduring practical problem lies in the estimation of these entropies from observed data. Th...
متن کاملCharacterizations Using Entropies of Records in a Geometric Random Record Model
Suppose that a geometrically distributed number of observations are available from an absolutely continuous distribution function $F$, within this set of observations denote the random number of records by $M$. This is called geometric random record model. In this paper, characterizations of $F$ are provided in terms of the subsequences entropies of records conditional on events ${M geq n}$ or ...
متن کامل